85 research outputs found
The supervised hierarchical Dirichlet process
We propose the supervised hierarchical Dirichlet process (sHDP), a
nonparametric generative model for the joint distribution of a group of
observations and a response variable directly associated with that whole group.
We compare the sHDP with another leading method for regression on grouped data,
the supervised latent Dirichlet allocation (sLDA) model. We evaluate our method
on two real-world classification problems and two real-world regression
problems. Bayesian nonparametric regression models based on the Dirichlet
process, such as the Dirichlet process-generalised linear models (DP-GLM) have
previously been explored; these models allow flexibility in modelling nonlinear
relationships. However, until now, Hierarchical Dirichlet Process (HDP)
mixtures have not seen significant use in supervised problems with grouped data
since a straightforward application of the HDP on the grouped data results in
learnt clusters that are not predictive of the responses. The sHDP solves this
problem by allowing for clusters to be learnt jointly from the group structure
and from the label assigned to each group.Comment: 14 page
Hamiltonian Latent Operators for content and motion disentanglement in image sequences
We introduce \textit{HALO} -- a deep generative model utilising HAmiltonian
Latent Operators to reliably disentangle content and motion information in
image sequences. The \textit{content} represents summary statistics of a
sequence, and \textit{motion} is a dynamic process that determines how
information is expressed in any part of the sequence. By modelling the dynamics
as a Hamiltonian motion, important desiderata are ensured: (1) the motion is
reversible, (2) the symplectic, volume-preserving structure in phase space
means paths are continuous and are not divergent in the latent space.
Consequently, the nearness of sequence frames is realised by the nearness of
their coordinates in the phase space, which proves valuable for disentanglement
and long-term sequence generation. The sequence space is generally comprised of
different types of dynamical motions. To ensure long-term separability and
allow controlled generation, we associate every motion with a unique
Hamiltonian that acts in its respective subspace. We demonstrate the utility of
\textit{HALO} by swapping the motion of a pair of sequences, controlled
generation, and image rotations.Comment: Conference paper at NeurIPS 202
Dynamic Trees:A Structured Variational Method Giving Efficient Propagation Rules
Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorised distribution over node states. Such a distribution seems unlikely in the posterior, as nodes are highly correlated in the prior. Here a structured variational approach is used, where the posterior distribution over the non-evidential nodes is itself approximated by a dynamic tree. It turns out that this form can be used tractably and efficiently. The result is a set of update rules which can propagate information through the network to obtain both a full variational approximation, and the relevant marginals. The propagation rules are more efficient than the mean field approach and give noticeable quantitative and qualitative improvement in the inference. The marginals calculated give better approximations to the posterior than loopy propagation on a small toy problem
- …